DOCUMENT RESUME 



ED i59 804 



FL 021 351 



AUTHOR 
TITLE 



INSTITUTION 

PUB DATE 

NOTE 

AVAILABLE FROM 



PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



Augustin, Marc 

Bilingual Education through Architectural Mathematics 
(Project beam), Community School District ?• 
Transitional Bilingual Education Grant T003A90275, 
1991-92, Final Evaluation Profile, OREA Report, 
New York City Board of Education, Brooklyn, NY, 
Office of Research, Evaluation, and Assessment, 
7 Jul 92 
15p, 

Office of Research, Evaluation, and Assessment, New 
York City Public Schools, 110 Livingston St,, Room 
732, Brooklyn, NY 11201, 
Reports - Evaluative/Feasibility (142) 

MFOl/PCOl Plus Postage, 

Admission Criteria; "Architectural Education; 
Attendance Patterns; Behavioral Objectives; 
''^Bilingual Education; Curriculum Development; Dropout 
Prevention; Educational Objectives; Elementary 
Education; Elementary School Students; ^English 
(Second Language); Enrollment; Mainstreaming; 
''Mathematics Instruction; '''Native Language 
Instruction; Parent Participation; Program Design; 
Program Evaluation; Research Methodology; Spanish 
Speaking; Staff Development; Staff Utilization; 
Student Characteristics ; Teachers ; Transitional 
Programs 

New York City Board of Education; '^'Project BEAM NY 



A bilingual education program in two New York City 
elementary schools is evaluated. The program served 369 students, 
most of them limited-English-proficient native Spanish-speakers, 
providing supplementary instructional services in English as a Second 
Language (ESL) , native language arts, and mathematics in the context 
of architecture. The report summarizes the program's staffing, 
specific behavioral objectives and results in those content areas, 
attendance and persistence patterns, student grade retention, 
services for students with special needs, mainstreaming, staff 
development, curriculum development, and parental involvement, and 
includes: one brief case study. Notes on methods of data collection 
and analysis for the evaluation and a list of instructional materials 
used are appended. (MSE) 



Reproductions supplied by EDRS are the best that can be made - 
from the original document. * 



I I ■ I I 

I I ■ I I 



Bilingual Education Through Architectural Mathematics 
(Project BEAM) 
Community School District 7 
Transitional Bilingual Education Grant T003A90275 

1991-92 

FINAL EVALUATION PROFILE 



••PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER {ERiC). " 



U.S. DEPARTMENT Of EDUCATIftw 

I EOUcrr 0^"°"" -^-«« 



BESTcepymunMil 



NEW YORK CITY BOARD OF EDUCATION 



H. Carl McCall 
President 

Irene H. Impellizzeri 
Vice President 

Carol A. Gresser 
Westina L. Matthews 
Michael J. Petrides 
Luis 0. Reyes 
Ninfa Segarra 
Members 

Keysha Z. McNeil 
Student Advisory Member 



Joseph A. Fernandez 
Chancellor 



DIVISION OF STRATEGIC PLANNING/'RESEARCH & DEVELOPMENT 

Robin WILlner 
Executive Director 



n.a«!l^n»^« s**/"^ ^''^ Educauon not to diicnmm.a: on ihc baiu of ncc. color, creed, rnhgion 

lSrn^*J^^.S^^^ P^^*": cr.virauncr.i fr« of sexual himimcnu m itcuii^bTliw 

Equal Oppo«umiy. UoLvv,|«« ScS^loom 601. Br^aJyn^cTYodJT^Td^S^^ 



ACKNOWLEDGMENTS 



This report has been prepared by the Bilingual, Multicultural, and Early Childhood Evaluation Unit of 
the Office of Research, Evaluation, and Assessment. Thanks are due to Marc Augustin for collecting the data 
and writing the report. 

Additional copies of this report are available from: 

Dr. Tomi Deutsch Berney 

Office of Research, Evaluation, and Assessment 

New York City Public Schools 

110 Livingston Street, Room 732 

Brooklyn, NY 11201 

(718) 935-3790 FAX (718) 935-5490 



FOREWORD 



The body of this report is preceded by an Extract which presents an overview of salient points of 
the project: funding cycle; enroUment Qgures; background of students served; admission criteria; and 
programming features, strengths, and limitations, including the outcome of all objectives. The extract also 
presents the conclusions drawn by the Office of Research, Evaluation, and Assessment (OREA) about the 
program and its recommendations for program improvement. 

The extract is followed by the body of the report, titled Program Assessment . This includes such 
information as staffing, program implementation, and outcome and implementation objectives. 
Instructional objectives are presented first, followed by noninstructional objectives. The report then 
addresses those aspects of programming mandated by Htle VII regulations that do not have specifically 
stated objectives. This may be information on attendance and dropout rate, grade retention, 
mainstreaming, referrals out of the program to meet special needs of the students, and withdrawals. A 
case histoiy concludes the report. 

Data for this profile were collected and analyzed using a variety of procedures, which are 
described in A ppendix A following the text. 



Office of Research, Evaluation, and Assessment 
New York City Public Schools 

E.S.E.A. Title VII Evaluation Profile 

Bilingual Education Through Architectural Mathematics 

(Project uEAM) 

Community School District 7 Bronx 
Transitional Bilingual Education Grant T003A90275 

1991-92 

EXTRACT 

PROJECT DIRECTOR: Ms. Yvette Boronow t UNDING CYCLE: Year 3 of 3 

SITCS 



School Grade I .c\x'is Enrollment* 

P.S. 49X 2-5 169 

P.S. 156X :.5 200 

*The project enrolled 369 students (121 more than in ihc prcs ious year). Male students numbered 160, female 
students 169; gender was not stated for 40. 

STUDENT BACKGROUND 



Native Language Number of Students (\)untries of Origin Number of Students 

Spanish 340 United States 184 

English 29 Dt^minican Republic 66 

Puerto Rico 57 

Mexico 29 

Honduras 9 

Ixuador 6 

(Guatemala 5 

111 Salvador 3 

Other 4 

Unrep<^rted 6 



Median Years of Education in Native Country: 2.0; in the United States: 3.0 
Percentage of Students Eligible for Free Lunch Program: f>^< 3 



ADMISSION CRFFERIA 

The project admitted students who scored at or bckrw the 40th percentile on the Language Assessment 
Battery (LAB) and were thus designated as being of limited I nglish proficiency (LEP). A number of English 
proficient (EP) students who would profit from the program were also admitted. 
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PROGRAMxMING 



Design Features 

Project BEAM was designed to provide supplementary instructional services to limited English proficient 
(LEP) students. Using architecture as a setting for instruction, the project was to offer courses in English as a 
Second Language (E.S.L.), Native Language Arts (N-LA.), and mathematics. Parental participation and staff 
development were designed to be an integral part of this project. 

Capacity building. Title VII funding was renewed for two years, starting next year. When this renewal of 
fundmg runs out, C.S.D. 7, which has been contributing continuously to the funding of the project, is planning to 
take it over. 



Strengths and Limitations 

Project BEAM carried out those activities it had proposed in its design. The project director reported 
that the architectural focus and students' creation of model bridges, houses, and other structures developed 
greater self-awareness and self-esteem in participants. 



CONCLUSIONS AND RECOMMENDATIONS 

Project BEAM was fully implemented. It met its objectives for E.S.L., curriculum development, and 
parental involvement. Contrary lo Lst year, the project did not meet its objective for N.L.A. this year. The 
project met its objective for demonstrating knowledge and appreciation of architecture, but not the objective for 
the content area subject of mathematics. Project BEAM met its staff development objectives for instructional use 
of architecture and staff participation in workshops and conferences, but failed to meet the staff development 
objective for enroUment in university courses. The project came close, but did not meet, its attendance objective. 

The conclusions, based on the findings of this evaluation, lead to the following recommendations: 

• Explore ways of increasing N.L.A. skills. 

• Augment instruction in mathematics, possibly through peer tutoring, mdividualized instruction, and/or 
an after-school tutorial program. 

• Encourage program teachers to enroll in university courses. 

• Involve resource teachers in working with groups of students on the importance of good attendance. 
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PROGRAM ASSESSMENT 



STAFFIN G 

Title VII Staff (Total 3) 
Title 

Project Director 
Resource Teacher 
Resource Teacher 



Degree Language Competencies 



M.A., RD. 

B.A. 

M.A. 



Spanish 
Spanish 
Spanish 



Title VII Funding Other Funding 

Part-time C.S.D. 7 
Full time 
Full time 



Other Staff Working With Project Students (Total 12) 

Title Degree CertiGcation 

Teachers 12 M.A. 12 Bil. Common Branches 12 
All staff had certiQcation in areas they served. 



Language Competencies and Teaching 
Communicative Proficiencies (TP/CP) 

Spanish TP 12 



IMPLEMENTATION AND OUTCOMES (objectives prefaced by •) 

Project beam's resource teachers and a licensed architect contracted by Learning through an Expanded 
Arts Program (LEAP) provided participating students with instruction in English, Spanish, mathematics and 
architecture. Resource teachers coordinated topics with non-project classroom teachers to aisure congruence. 
The resource center was also open to visits from other classes. 



English as a Second Languap:e TE.S.L.) 

Project students received five periods of E.S.L. instruction each week. Participating students, with their 
classroom teacher, attended the Project BEAM resource center, where they were assisted by a bilingual project 
resource teacher. Lessons were based on levels of proficiency. Instructional strategies included peer tutoring, 
dramatization, grouping, and cooperative learning. For a list of instructional materials see Appendix B. 

• As a result of participating in the program, students will demonstrate achievement beyond expectation 
in EngUsh as a Second Language (p<.05) in tho N.C.E.S as measured by the English version of the 
Language Assessment Battery (LAB). 

Evaluation Instrument: Language Assessment Battery (LAB)** 



Teaching Proficiency (TP): Competent to teach in this language. 
Communicative Proficiency (CP): Conversational capability only. 

OREA used a gap reduction design to evaluate the effect of supplementary instruction on project students' 
performance on the LAB. Since all LEP students in New York City are entitled to such instruction, no valid 
comparison group exists among these students, and OREA used instead the group on which the LAB was 
normed. Test scores are reported in Normal Curve Equivalents (N.CE.s), which are normalized standard scores 
with a mean of 50 and a standard deviation of 2L06. It is assun>ed thai the norm group has a zero gain in 
N.C.E.S in the absence of supplementary instruction and that participating students' gains are attributable to 
project services. 



Pretest: May 1991; posttest: May 1992. 

Number of students for whom pre- and posttest data were reported: 262 

Mean gain: 6.1 N.C.E.s (s.d.=16.6) 

Mean gain is statistically significant (/=5.94, p<.05). 

Project met E.S.L. objective. 



Native Language Arts (N.L.A.) 

Estimated percentage of students lacking literacy skills in native language: less than 1 percent. 

Participating students received five periods of N.L.A. instruction weekly. Some instructional methods 
included peer tutoring and cooperative learning. By taking notes and reading (silently and aloud), students 
developed listening comprehension and fluency in their native language. For a list of instructional materials see 
Appendix B. 

• As a result of participating in the program, students will demonstrate achievement beyond expectation 
(p<.05) in N.C.E.s as measured by the Spanish version of the Language Assessment Battery (LAB). 

Evaluation Instrument: Pretest, Reading Subtest of Spanish LAB, May 1991; posttest, Examen de Lectura en 
Espanol (ELE), May 1992.* 

Pretest: May 1991; posttest: May 1992. 

Number of students for whom pre- and posttest data were reported: 71 

Mean gain: -6.6 N.C.E.S (s.d.=20.4) 

Mean gain is statistically significant (f=-2.75, j7<.05) 

Project did not meet N.LA. objective. The project director stated that there would be more reading 
periods in the next program year. 



Content Area Subjects 

Project BEAM students were enrolled in courses in mathematics (including architecture), science, and 
social studies. The project proposed objectives for mathematics only. 

Project BEAM exposed students to mathematical concepts by providing them with hands-on activities in a 
stimulating learning environment focusing on architecture. In early grades, topics introduced included measuring 
the height of people; using scale numbers; and making and using tape measures. In later grades, topics included 
solving word problems and dimensions and formulas for areas and volumes. 

Students had five periods of instruction in architectural mathematics weekly. For a list of instructional 
materials see Appendix B. 

• As a result of participating in the program, students will demonstrate achievement beyond expectation 
(p<S}5) in N.CE.S as measured by the New York City Mathematics Test for program evaluation. 
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Details for the evaluation design using the spring 1991 Spanish LAB administration as a pretest and the spring 
1992 ELE administration as a posttest will be forthcoming. 



Evaluation Instrument: Metropolitan Achievement Test in Mathematics (MAT-Math) 
Pretest: April 1991; posttest: April 1992. 

Number of students for whom pre- and posttest data were reported: 199 

Mean gain: -4^ N.CE.S (s.d.=14.9) 

Mean gain is statistically significant (f=-4.27, /?<.05). 

Project did not meet mathematics objective. The project director reported that more staff development 
in mathematics instruction would be offered and she anticipated improvement in the next program year. 

• As a result of participating in the program, 80 percent of the students will demonstrate knowledge 
and appreciation of various architectural techniques. 

Evaluation Indicator: teacher judgment. 

Number of students for whom data were reported: 247 

Percentage of students who showed progress: 93.5 

Project met objective for demonstrating knowledge and appreciation of architecture. 

Attendance 

• As a result of participating in the program, there will be a significant gain in student attendance. 

ATTENDANCE RATES 

School 1991-92 1990-91 Difference 

P.S. 49 92.5 90.9 1.6 

P.S. 156 95.0 94.1 0.9 

Although students' attendance increased, the gains were not statistically significant. 

Project did not meet attendance objective. The project director stated that the project would offer more 
parental workshops on the importance of attendance, in the hope of influencing students. 

Dropout Prevention 

As in the previous year, no project students dropped out. The project did not propose any objectives for 
dropout prevention. 

Grade Retention 

Eight students were retained in grade. This represented a 50 percent decrease from the previous year. 
No objective was proposed for reducing grade retention. 



' 10 



Students With Special Academic Needs 



Referral to special e ducation. Three students were referred to special education. In accordance with 
New York City Public Schools guidelines, Project BEAM stafif followed the procedures established by C.S.D. 7 for 
referring students to special education. The School-based Support Team (S.B.S.T.) evaluated students before 
placement. At P.S. 156, one S.B.S.T. member spoke the language of the students being evaluated. None of the 
S.B.S.T. members spoke the students' language at P.S. 49. The resource teacher at this school, however, served as 
an interpreter for Spanish-speaking students. 

Remedial programs. Twelve students were referred to remedial programs. OREA did not have figures 
for the previous year. 

Gifted and talented piograms. No students were referred to gifted and talented programs. OREA did 
not have figures for the previous year. 



Mainstreaniing 

One student (0.3 percent of participants) was mainstreamed. OREA did not have mainstreaniing figures 
for the previous year. 



Staff Development 

• Participating teachers will demonstrate a greater ability to use various forms of architecture in ihcir 
E.S.L. and Spanish reading instruction. 

As a result of coordinating lessons with the resource teachers and attending field trips, participating 
teachers were better able to use architectural concepts to stimulate students' interest and further develop tiicir 
English and Spanish reading, writing, and listening skills. 

Project met objective for staff development in instructional use of architecture. 



• Sixty percent of program teachers will have enrolled in at least one university course each scniosicr. 

Tuition reimbursement was provided for all participating teachers who requested it. The project dirccior 
reported that only 30 percent of the teachers enrolled m university courses, smce most already had a niasior s 
degree. 

Project did not meet objective for enrollment in university courses. The project director will cniphasi/c 
the importance of this staff development objective in the next program year. 



• Sixty percent of program teachers will have participated m teacher training workshops and 
conferences. 

Ail Project BEAM teachers participated in districtwide staff development conferences and a number o\ 
on-site Project BEAM workshops. In addition, the project director sent informal letters to disseminate 
information to teachers about materials, activities, curricula, etcetera. 

Project met objective for staff participation in workshops and conferences. 
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Curriculum Development 

• A draft of a manual containing learning activities and methods to be used for E.S.L./Spanish reading 
will be prepared by the participating staff and teachers. 

Project staff and teachers drafted a curriculum guide for E.S.L./Spanish reading lessons based on 
architectural concepts. 

Project met curriculum development objective* 



Parental Involvement 

• Sixty percent of the parents of participating students will have participated in program-related 
activities. 

Project BEAM provided workshops, meetings, field trips, and an art exhibit for the parents of 
participating students. Project attendance records mdicated that over 60 percent of the parents participated. 

Project met objective for parental involvement in program-related activities. 



• Sixty percent of the parents of participating students will have participated in E.S.L. and Spanish 
Reading through Architectural Mathematics activities. 

Project records showed that over 60 percent of the parents participated in E.S.L. and Spanish reading 
through architectural mathematics activities. 

Project met objective for parental involvement in £,S,L, and Spanish reading activities. 



CASE HISTORY 

D.C. of P.S. 49 was i shy and timid child In the school since kindergarten, he was now in the fourih 
grade. In the past, D.C. was usually reluctant to perform tasks without prior approval. The program has helped 
him to follow dkections, and he has shown a wilUngness to participate more often. He is also more 
communicative, and his mathematics and reading scores in both English and Spanish have improved. 



APPENDIX A 



DATA COLLECTION AND ANALYSIS 



COLLECTION 

OREA evaluation consultants visit sites and interview key personnel. The project director gathers data 
and, with the consultant, completes forms (as shown below) as necessary. 

St udent Data Form 

This one-page form is filled out by staff for each participating and mainstreamed student. OREA gathers 
data from this form on backgrounds, demographics, academic outcomes, attendance, referrals, and exit from the 
program. 

Project Director's Questionnaire 

The Project Director's Questionnaire includes questions on staff qualifications, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview gathers information on program and student or staff characteristics not supplied by the 
Project Director's Questionnaire. The interview also allows project staff to offer qualitative data or amplify 
responses to the questionnaire. 

Citywide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test scores are reported in Normal Curve Equivalents (N.C.E.s). N.C.E.s are 
normalized standard scores with a mean of 50 and a standard deviation (s.d.) of 21.06. They constitute an equal- 
interval scale in which the distance is the same between any two adjacent scores. A gain of 5 N.C.E.S is the same 
whether it is at the lower or the higher end of the scale. N.C.E.S can be used in arithmetic computations to 
indicate group progress. (Percentile scales, although more familiar to many, are unsuitable for such computations 
since they are not equal-inter\'al.) 

Likert-Type Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to five, are used in a 
variety of ways. They examine student attitudes (i.e., toward school and career, native language use, and native 
and mainstream cultures). They also assess staff and parent attitude and reactions to workshops and other 
activities. 



ANALYSIS 

Gap Reduction Evaluation Design 

OREA uses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project comparison group is available in New York City, where all students of limited English proficiency 
(LEP) are entitled to receive supplementary services, OREA compares the progress of participating students with 
that of the group on which the test was normed. It is assumed that the norm group would show a zero gain in 
the absence of instruction, and gains made by project students could be attributed to project services. Scores are 
reported in Normal Curve Equivalents (N.C.E.s), which are normalized standard scores with a mean of 50 and a 
sundard deviation of 21.06. (See "Citywide Test Scores" above.) 

To test whether pre/posttest gains are greater than could be expected by chance alone, OREA uses a h 
test. To test whether a difference between two proportions (e.g., program and mainstream attendance rates) is 
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greater than could be expected by chance, OREA uses a z-test and reports the differences between the two 
proportions. The level of significance is set at .05 for all tests. 

Techniques For Minimizing Error 

The evaluation procedures minimize error by providing for proper administration of evaluation 
instruments through a combination of testing at 12-month intervals, appropriate analysis procedures 
and reporting. 

Instruments of measurement include the LAB (see above), the Degrees of Reading Power (D.R.P.) test, 
the MetropoUtan Achievement Test-Mathematics (MAT-Math), EI Examen de Lectura en Espanol (ELE), 
Likert-type scales (see above), and project-developed tests. Except for Likert scales and project-developed tests, 
these instruments are scored on a citywide basis at the Scan Center of the New York City Public Schools. 



APPENDIX B 
INSTRUCTIONAL MATERIAI^ 



E.S.L- 



BEAM Curriculmn Guide 

E,S,L Activity Kit 

Open the Light 

Architecture series 

Where the Wild Things Are 

Complete ESLIEFL Resource Book 



Project BEAM 
E. Claire 

Coreen Carruthers 
D. MacCauly 
Maurice Sendak 



CS.D. 7 
Prentice Hall 
Addison Wesley 
Houghton Mifflin 
Harper & Row 
National Textbooks 



N.L.A. 

BEAM Curriculum Guide 
Campanitas de oro 
Mil maravillas 
Individual Reader 



Project BEAM 
Spanish Big Books 
Spanish Reading Series 



CS.D. 7 

Lectorum Publishing Co. 
MacMillan Publishing Co. 
Modern Curriculum Press 



Content Area Subjects 

BEAM Manuals Project BEAM CS.D. 7 

Mathematics Holt, Rinehart & Winston 

Ciencias SUver Burdett 

Ciencias Holt, Rinehart & Winston 

Geografia de oyer y hoy Silver Burdett 



